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ABSTRACT: Exoskeletons are gaining attention as a potential solution for addressing low back injury in the 
construction industry. However, use of active back-support exoskeletons in construction can trigger unintended 
consequences which could increase mental workload of users while working with exoskeletons. Prolonged increase 
in mental workload could impact workers’ wellbeing and productivity. Prediction of mental workload during 
exoskeleton-use could inform strategies to mitigate the triggers. This study investigates a machine-learning 
framework for predicting mental workload of workers while using active back-support exoskeletons for 
construction work. Laboratory experiments were conducted wherein Electroencephalography (EEG) data were 
collected from participants wearing active back-support exoskeletons to perform flooring task. The EEG data 
underwent preprocessing, including band filtering, notch filtering, and independent component analysis, to remove 
artifacts and ensure data quality. A regression-based Long Short-Term Memory network was trained to forecast 
future time steps of the processed EEG data. The performance of the network was evaluated using root mean 
square error (RMSE) and r-squared (R’). A RMSE of 0.1527 and R° of 0.9665 indicating good fit and strong 
correlation, respectively, were observed between the predicted and actual EEG data. Results of the comparison 
between the actual and predicted mental workload also show strong correction with an R° of 0.8692. The findings 
motivate research directions into real-time monitoring of mental workload of workers during exoskeleton-use. The 
study has significant implications for stakeholders, enabling them to gain a deeper understanding of the impact of 
mental workload while using exoskeletons thereby providing opportunities for mitigation. 
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1. INTRODUCTION 


The prevalence of work-related musculoskeletal disorders (WMSDs) among the construction workforce is a 
growing concern in the construction industry. The US Bureau of Labor Statistics reports that workers in the 
construction industry are 1.23 times more likely to sustain WMSDs compared with workers in other industry 
sectors (BLS, 2020). The same report explains that the back is the one of the most affected body parts. Construction 
workers, such as floor layers, suffer from back injuries at 1.7 times workers in other industry sectors. For example, 
floor layers experience back injuries at the rate 22.5 MSDs per 10,000 full-time workers, and this has been known 
to result in an average of 26 days’ work absence. Exoskeletons are increasingly being perceived as a solution to 
WMSDs. Exoskeletons, such as back-support exoskeletons, are wearable devices designed to support or augment 
users’ back while performing work (Gonsalves et al., 2023; Ogunseiju et al., 2022). Exoskeletons are classified as 
passive and active depending on their mode of augmentation. Passive back-support exoskeleton, while less costly 
than active back-support exoskeletons, provide support to the back using dampers and springs. Whereas active 
back-support exoskeletons provide support to the back using electrical motors — this makes active back-support 
exoskeletons bulkier. These devices have been shown to reduce risks factors of back injuries by reducing muscle 
activity (Theurel et al., 2018), range of motion (Cumplido-Trasmonte et al., 2023), body discomfort (Gonsalves et 
al., 2021; Kim et al., 2019), and rate of exertion (Alemi et al., 2020; Baltrusch et al., 2021). Despite these benefits, 
studies have shown that exoskeleton-use in construction can trigger difficulty working in confined spaces 
(Nussbaum et al., 2019), fall risks due to the weight of the device (Alabdulkarim et al., 2019; Kim et al., 2019; 
Massardi et al., 2023), discomfort to body parts (Gonsalves et al., 2023; Gonsalves et al., 2021), restrictions in 
movement (Fox et al., 2020; Poliero et al., 2020), catch and snag risks (de Looze et al., 2016; Kim et al., 2019), 
and thermal discomfort (Liu et al., 2021). The devices could also be challenging to adjust to fit (Gonsalves et al., 
2023; Gorgey, 2018). Moreover, unequal loading and balancing of body parts due to improper adjustment can 
cause users to be more aware of the device than their task and surrounding, which could increase workers’ mental 
workload (Bequette et al., 2020; Marchand et al., 2021). 


Prolonged increase in mental workload can result in distraction, emotional distress, anxiety, and stress, which have 
downstream implication on workers’ overall well-being and performance. Real-time monitoring of workers’ 
mental workload during exoskeleton-use could inform strategies to reduce the triggers. However, scarce efforts 
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have been made to investigate models for predicting workers’ mental workload during exoskeleton-use. 
Electroencephalogram (EEG) can be used to measure brain activity corresponding to mental workload. EEG 
signals are widely used for inferring mental workload, due to the high temporal resolution, convenience, and cost- 
effectiveness of the supporting devices. Machine learning techniques, particularly deep learning, provides 
opportunities for extracting insightful features from EEG data that could be used for predicting mental workload. 
Long Short-Term Memory network, a variant of recurrent neural network, can learn long-term dependencies 
between time steps of data and predict future time-series sequences of the data. Long Short-Term Memory (LSTM) 
network has been used for sequential learning tasks like construction equipment activity analysis (Hernandez et 
al., 2019), construction workers’ safety harness usage (Guo et al., 2023), mixed reality learning environment 
(Ogunseiju et al., 2023) and, fatigue detection and early warning system (Liu et al., 2020) that need historical time- 
series data in the decision-making process. Therefore, this study investigates the extent to which workers’ mental 
workload due to exoskeleton-use can be predicted from EEG data using Long Short-Term Memory network. Using 
a case-study of a flooring task, this paper presents a comparison of the actual and predicted mental workload due 
to performing work with an active back-support exoskeleton. The results of this study contribute to scarce 
knowledge on the unintended consequences of using wearable devices such as exoskeletons for construction work. 


2. BACKGROUND 
2.1 Mental Workload Evaluation with EEG 


Mental workload are the mental resources required for task execution (Chen et al., 2017). A previous study (Fan 
& Smith, 2017) has shown that mental workload is associated with task demand and performance. Mastropietro et 
al. (2023) showed that low mental workload (underload) and high mental workload (overload) can negatively 
affect the task being executed leading to increase in rate incidence of errors. Chen et al. (2016) opined that when 
a person places too much attention on a task, the individual has less attention to focus on other stimuli. In the 
context of this study, exoskeleton-use may demand attention, thereby reducing the mental resources that may be 
required for the task or being aware of the user’s surrounding to prevent fall risks, and catch and snag risks 
(Gonsalves et al., 2023; Zhu et al., 2021). These risks can impact the mental workload resulting in exoskeleton 
users being stressed or distracted, thus retarding their productivity and safety. This makes prediction of mental 
workload a major interest in ergonomics (Young et al., 2015). Over the years, subjective and objective methods 
have been employed to infer mental workload. Subjective methods include the use of questionnaires such as NASA 
Task Load Index and Subjective Workload Assessment Technique. On the other hand, objective methods include 
the use of data collection instruments such as functional magnetic resonance imaging-fMRI, and 
electroencephalography (Ryu & Myung, 2005). However, EEG has been touted as one of the most suitable devices 
for measuring brain activities to infer mental workload (Chen et al., 2016; Qin & Bulbul, 2023). 


Borghini et al. (2014) estimates mental workload using theta-to-alpha brain waves ratio from EEG data. Similarly, 
another study (Missonnier et al., 2006) indicated that using EEG signals, an increase in mental workload is noticed 
when there is a decrease in alpha brain waves (8-13Hz) and increase in the theta brain waves (4-8Hz) during the 
execution of specific tasks. In the construction industry context, EEG has been used in some studies (Chen et al., 
2016; Chen et al., 2017; Qin & Bulbul, 2023; Yang et al., 2023) to estimate mental workload. For instance, EEG 
was used to estimate the mental workload of construction workers to on-site safety conditions (Chen et al., 2016). 
Engagement index, time-frequency indicator in EEG, was used to assess the mental workload of workers when 
exposed to construction vulnerabilities. Chen et al. (2017) used EEG approach to measure task mental load of 
construction workers based on the power spectral densities of major frequency bands. In addition, the effect of 
distractions in construction work zones on drivers’ mental workload was measured using EEG (Yang et al., 2023). 
Despite these efforts and unintended consequences of exoskeletons, there are scarce studies on predicting mental 
workload due to exoskeleton-use on construction sites. 


2.2 Machine Learning for Mental Workload Prediction 


With machine learning techniques, EEG data can be transformed into frequency domain representations which can 
enable extraction of brain rhythms. For instance, Jebelli et al. (2018a) used support vector machine, a supervised 
machine learning technique, to classify stress levels of construction workers. However, the use of supervised 
learning technique requires hand-crafting of features which could be labor-intensive and may be insufficient to 
support real-time monitoring of mental workload (Wang et al., 2023). Deep learning techniques, such as 
convolutional neural networks (CNN), have been used to extract intrinsic features from time-series data for 
recognizing occupational stress, fatigue and mental workload (Jebelli et al., 2019; Mehmood et al., 2023; Qin & 
Bulbul, 2023). Recurrent neural network, a class of CNN, is widely used for forecasting time-series data such as 
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brain activity. Recurrent neural network, such as Long Short-Term Memory (LSTM), has been noted to perform 
better in learning time-series data due to its high prediction accuracy and ability to overcome problems of 
overfitting (Wang et al., 2020). Furthermore, LSTM can solve the problem of gradient exploding and vanishing 
when processing large sequential data (Hochreiter & Schmidhuber, 1997). Jaiswal et al. (2023) noted that LSTM 
model performed better than other models in predicting cognitive fatigue. Moreover, in their study, LSTM 
eliminated the need for extensive data preprocessing and feature extraction which could have resulted in loss of 
useful information in EEG data. Also, Liu et al. (2022) used LSTM for detecting fatigue of drivers. In construction, 
Qin and Bulbul (2023) used LSTM for predicting the mental workload of workers while using augmented reality 
head-mounted display for construction assembly. Despite these possibilities, limited studies have harnessed LSTM 
for predicting mental workload associated with exoskeleton-use for construction work. 


3. METHODOLOGY 


This section, including Figure 1, describes the procedure employed to achieve the objective of the study including 
the experimental design to collect brain activity of participants performing flooring task with an exoskeleton, 
preprocessing of the brain activity data, and prediction of mental workload with the data. 


Prediction of EEG 


Experimental Design Data Collection Data Processing Mental Workload 
T T T 
| ; | 
A @ Participants (n = 8) @EEG device @ EEGLAB ® LSTM network 
| 
è Performance Evaluation 
Í * Lifting 
CA ® Task (Flooring Task)==>} + Placing 
* Installation 


* No Exo condition 
® Conditions =»| + Passive Exo condition 
* Active Exo condition 


Fig. 1: Overview of Methodology. 


3.1 Experimental Design and Data Collection 


Eight male graduate students (n = 8) where recruited to perform a flooring task with an active exoskeleton. Similar 
sample sizes have been used by previous studies (Wei et al., 2020). None of the participants reported any prior 
musculoskeletal injury that would impact their participation in the study. The active exoskeleton used for the study 
is the Cray X shown in Figure 2. The Cray X, from German Bionic, weighs 7kg and can provide a lifting support 
of about 30kg. Cray X consists of a frame and strap pads of different sizes for the legs, chest, shoulders, and waist. 
The frame includes a 40V battery and motor. The exoskeleton provides different levels of support for bending, 
lifting, placing, and walking. 


Hot-swappable 40 
V battery at the Vest 
back. + * 


User interface 
for process 
optimization 


¥ 


Fig. 2: Active (CrayX) back-support exoskeleton. 
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The flooring task involved lifting, placing and installing 20 floor tiles in each bay of a wooden frame comprising 
of six bays. Each bay can fit 20 floor tiles (see Figure 3). The participants were asked to lift and place 20 timber 
tiles (10kg) beside each bay, and subsequently install the stacked tiles in each bay. Each tile weighs 0.5kg. A cycle 
of flooring task includes lifting, placing, and installation of the timber floor tiles (20) in each bay. The task 
comprises of six cycles given that the participants installed the tiles in six bays. 


Fig. 3: Experimental layout of the simulated flooring task. 


Prior to commencing the tasks, the participants received instructions on how to perform the task. The participants 
performed the aforementioned flooring task with an active exoskeleton. During these conditions, the participants 
wore an EEG cap. The EEG device records electrical activity of the brain through contact between electrodes 
embedded in various portions of the cap and the scalp. The brain produces electrical signals of brain waves at 
different frequencies such as delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-13Hz), beta (13-30Hz), and gamma (>30Hz) 
(Chen et al., 2023). The delta, theta, alpha, beta and gamma bands correspond to deep sleep, powered thinking, 
alertness, concentration, and attentional processing, high mental activity, and information processing respectively 
(Ke et al., 2021). This study utilized a 14-channel EEG device measuring brain activity at 256Hz. 


3.2 Data Preprocessing 


EEG data are susceptible to contamination from intrinsic and extrinsic artifacts, particularly when subjects are 
engaged in physical activities like construction work (Jebelli et al., 2018b). These artifacts impact the quality of 
the signal. Intrinsic artifacts are triggered by movements such as eye blinking and muscle movement, while 
extrinsic artifacts are caused by external influences such as noise from wires and electrode popping. This study 
used the framework proposed by Jebelli et al. (2018b) to reduce the artifacts in the EEG data obtained from the 
simulated task. The EEG data were fed into EEGLAB, a MATLAB toolbox for processing physiological data. The 
extrinsic artifacts were removed using a Bandpass filter with cut-off frequencies of 0.5 and 65 Hz (Jebelli et al., 
2018b). Another extrinsic artifact due to noise from wires was removed using a notch filter applied at a frequency 
of 60Hz. The intrinsic artifacts were removed using independent component analysis (ICA) (Mantini et al., 2008). 
The EEG data was decomposed using Extended Infomax method into 14 components, representing the 14 channels 
of the EEG device, and displayed using a scalp heatmap (Frølich & Dowding, 2018). Preprocessed data from eight 
channels (i.e., AF3, F3, P7, O1, O2, P8, F4, and AF4) were utilized for this study. The data points from the channels 
were split into training, validation, and testing, accounting for 70%, 10%, and 20% respectively. 


3.3 Prediction of EEG Data 
3.3.1 Long Short-Term Memory network 


LSTM network, deep learning artificial recurrent neural network variant (RNN), was leveraged in this study to 
forecast subsequent values of the preprocessed EEG data. LSTM takes cognizance of the changes that could occur 
as the user gets used to the use of the device over time. LSTM neural network processes data by iterating over 
current time steps and retaining useful information to help with the processing of new data points. The regression 
LSTM neural network consists of four layers: an input layer, the LSTM layer, the fully connected layer and a 
regression layer. The input layer accepts the input time-series data and transfers this to the LSTM layer. The LSTM 
layer comprises of a cell, an entry gate, an exit gate, and a forget gate. The cell stores long-term time-series data 
and uses the gates to control flow of the data within and out of the cell. The forget gate decides which information 
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should be ignored in the cell. The LSTM layer comprises of 128 hidden units. The number of hidden units 
determines how much information or data is learned by the layer. More hidden units could result in better results 
but are more likely to result to overfitting to the training data. The fully connected layer does the discriminative 
learning in the LSTM network. It learns weights that can identify features in the training data. The regression layer 
determines the performance metrics needed for the prediction task. The LSTM neural network is trained with a 
time-series sequence of EEG data, where the outputs are EEG values of subsequent time steps. To prevent 
overfitting and divergence of the training, the predictors and targets were normalized to zero mean and unit 
variance (Jebelli et al., 2018b). Hyperparameters have a significant impact on the performance of models. The 
network was trained with the Adam optimizer, an extension of stochastic gradient descent. In addition, 200 epochs, 
as well as a learning rate of 0.001 was used. 


3.3.2 Performance evaluation 


The performance of the LSTM model was evaluated using the Root Mean Square Error and R-squared. Root Mean 
Square Error (RMSE) is a standard statistical metric for computing accuracy. RMSE is generally used to evaluate 
the difference between the actual and predicted value from the model. RMSE is determined via equation 1, where 
Ajand Pj are the actual and predicted EEG datasets respectively, and n is the number of EEG datasets. The lower 
the RMSE, the better a model would fit a dataset. The R-Squared (R°) which describes the variance in the response 
of a regression model, was computed following Renaud and Victoria-Feser (2010). The R? value ranges from 0 to 
1. The higher the R? value, the better a model fits a dataset. 


RMSE = [Y -Ad?/n (1) 


3.4 Mental Workload 


The pre-processed EEG data (from Section 3.2) and the predicted EEG data from Section 3.3.1 were decomposed 
into frequency components to determine the power spectra of the data using Welch (1967)’s approach. The 
approach uses fast Fourier transformation with a Hamming window to determine the power spectral density of the 
EEG data. The relative band power of the windowed or segmented data in theta and alpha frequency bands were 
determined. Xing et al. (2020) mentioned that theta and/or alpha power are suitable indicators of mental workload. 
The mental workload of each segment was determined by dividing the absolute power in the theta band with the 
absolute power in the alpha band. The approximate spectral limits of these frequency bands are 4-8 Hz (theta) and 
8-14 Hz (Simon et al., 2011). 


4. RESULTS AND DISCUSSION 
4.1 Performance of the LSTM Model for Each Channel 


Table 1 shows the RMSE and R? scores for the EEG channels of one of the test participants. The RMSE values are 
less than 0.3 with the P7 and O1 channels having the lowest prediction errors. A previous study has indicated that 
a RMSE value closer to zero gives a better predictive power (Miyamoto et al., 2022). Similarly, the low RMSE in 
this study shows the high predictive power of the LSTM network in predicting mental workload. The R? scores of 
the channels are more than 0.9 which indicates close alignment or similarity between the predicted and actual EEG 
values. 


Table 1: RMSE and R? scores for all the EEG channels. 


AF3 F3 P7 Ol 02 P8 F4 AF4 Average 


RMSE 0.1174 0.1404 0.0772 0.0975 0.1700 0.1876 0.2090 0.2225 0.1527 


R? 0.9061 0.9144 0.992 0.9899 0.9941 0.9832 0.9755 0.9771 0.9665 
4.2 Mental workload 


4.2.1 Comparison between Predicted and Actual PSD 


Figure 4 shows the predicted and actual power spectral density of the AF3 and O2 channels for the data of one of 
the test participants. The predicted and actual data are the red and blue lines respectively. At less than 45Hz, both 
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figures show some consistency between the predicted and actual PSD values. 
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Fig. 4: Predicted and actual power spectral density of the AF3 channel (left) and O2 channel (right). 
4.2.2 Mental workload 


The extent to which mental workload due to exoskeleton-use can be predicted is illustrated in the scatter diagram 
in Figure 5. The plot has a R? score of 0.8692 indicating a strong correlation between the predicted and the actual 
mental workload. A previous study (Coulibaly & Baldwin, 2005) has shown that R? score between 0.8-0.9 is 
termed acceptable. The result suggest that it is possible to predict mental workload during exoskeleton-use for 
construction work. Previous studies have corroborated the assertion that mental workload can be predicted 
(Borghini et al., 2014; Missonnier et al., 2006; Qin & Bulbul, 2023). 
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0 
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Actual Mental Workload 


Fig. 5: Comparison of predicted and actual values of the mental workload. 


5. CONCLUSIONS, LIMITATIONS AND FUTURE WORK 


This study presents the extent to which mental workload due to exoskeleton-use can be predicted from EEG data 
using Long Short-Term Memory network. EEG data were obtained from an experimental study where participants 
performed flooring task with an active back-support exoskeleton. The data were preprocessed and trained with the 
Long Short-Term Memory network to identify unique features for forecasting brain activity. A comparison of the 
actual and predicted brain activity data indicates close consistency, with average root mean square error and r- 
squared of 0.1527 and 0.9665 respectively. Similar trends were observed in the comparisons of the predicted and 
actual power spectrums and mental workload. The results of this study contribute to scarce literature on the impact 
of unintended consequences of using exoskeletons for construction work. The study motivates investigations into 
the use of machine learning for real-time performance predictions of technological innovations on construction 
projects. The study may have been limited due to the sample size of eight participants which was used to train the 
deep learning algorithm. Training data from a larger sample could improve the performance of the model and its 
generalizability. This would be achieved by using time-series based data augmentation techniques such as scaling, 
permutation and generative adversarial networks. Future studies can compare the mental workload of no 
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exoskeleton and active exoskeleton conditions. In addition, further investigation on the suitability of other deep 
learning networks to identify the most suitable networks for predicting mental workload can be carried out. Besides, 
future studies can support mental workload prediction with the understanding of the risks influencing mental 
workload using subjective feedback that describes user experience of exoskeletons. 
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